Hybrid Email and Outpatient Clinics to Optimize Maintenance Therapy in Acute Lymphoblastic Leukemia

Acute lymphoblastic leukemia treatment includes an outpatient (OP)-based 2-year maintenance therapy (MT). Over an 8-year period, patients were transited from only OP to a hybrid e-clinic/OP-clinic model. Electronic and patient-held medical records of acute lymphoblastic leukemia patients 1 to 18 years old during MT were used to analyze demographics, drug doses, treatment response and cost. A survey evaluated family satisfaction with the hybrid service. Four hundred and seventy-eight children, all with at least 1 year of MT from March 13, 2014 to March 24, 2022 were grouped into 4 treatment eras, representing the transition from all OP (era 1) to the current hybrid MT practice (era 4). Cohort demographics were similar across all eras. With transition to era 4, OP visits decreased to a third (16 to 18/48 visits). Practice optimization in era 2 resulted in higher MT dose intensity in subsequent eras (era 1: median 82% [interquartile range, 63 to 97]; era 2: 93% [73 to 108]; era 3: 88% [68 to 106]; era 4: 90% [74 to 114], P<0·0001), with no differences in absolute neutrophil count or neutropenia-related toxicity (P=0.8). The hybrid service reduced MT expenses by ~50% and families (133/156, 85%) reported being very satisfied. Our experience indicates that a hybrid model is feasible, effective and less burdensome for patients and families.

the SARS-CoV-2 pandemic resulted in an enforced lockdown for many months.Patients were unable to travel, and hospital services were curtailed.All MT patients were moved to the e-clinic to ensure the continuation of therapy.We report on how this experience catalyzed the development of a hybrid approach to MT management at our center, decreasing the burden on families and the hospital while continuing to maintain drug dose titrations and optimal drug dosing.

Patients
All children treated for ALL at TMC on the ICiCLe-ALL-14 protocol 11 who commenced MT between March 13, 2014 and March 24, 2022 were eligible for the study.Patients without MT data for a minimum of 4 cycles starting from cycles 1 to 4 were not analyzed.Children were recruited with consent and with ethical approval (Institutional Review Board: EC/TMC/12/13).

Treatment
In the ICiCLe-ALL-14 protocol, the intensive treatment phase lasts for 24 to 29 weeks depending on risk stratification (Fig. 1A). 11MT is divided into eight 12-week cycles for a total of 96 weeks.Together with daily oral 6MP and weekly MTX, children receive intrathecal MTX once every cycle and oral trimethoprim-sulphamethoxazole prophylaxis.As per protocol, a CBC is done every 2 weeks or more frequently if required.Overall, a minimum of 48 "visits" are required either as OP or a combination of e-clinic and OP visits.MT OP and e-clinics are held twice weekly, jointly by a consultant and a resident.

Era Definition and Cohort Selection
For purposes of this study, the temporal changes in the delivery of MT were categorized into eras (Fig. 1B).In era 1 (May 1, 2014, to January 25, 2018)

Data Collection
Patient-held data sheets (see table, Supplemental Digital Content 1, http://links.lww.com/JPHO/A662)from eras 1 to 3 and e-clinic logbook entries from era 2, containing CBC and doses of 6MP and MTX, were digitized.Any missing data from handheld data sheets were completed by retrieval from electronic medical records.In era 3 and 4, a semiautomated spreadsheet (see spreadsheet, Supplemental Digital Content 2, http://links.lww.com/JPHO/A663) was created, individualized for each patient, stored on the hospital server, and accessible to both the e-clinic and OP teams.Data on MT information 12 (see spreadsheet, Supplemental Digital Content 3, http://links.lww.com/JPHO/A664) were extracted from these records.Along with OP records, emails from patients were screened to check for additional comments by families.Hospital admissions and registered postal codes of the patients were extracted from electronic medical records.The postal code was used to identify the home address 13 and compute the minimum distance between the patient's home to the hospital.

Dose and Response Analysis
In the ICiCLe-ALL-14 protocol, the recommended start doses of 6MP and oral MTX were 60 mg/m 2 /d and 20 mg/m 2 /wk, respectively (protocol-recommended doses).At each visit, drug doses were titrated based on blood counts.The targeted weighted means of prescribed antimetabolite doses over the course of MT were calculated for each patient and reported as a proportion of the protocolrecommended doses to represent dose intensities (DIs).DIs were determined individually for 6MP, MTX and for the antimetabolite combination (product of the DIs of 6MP and MTX).The interval (in weeks) between dose prescription visits (in-person or remotely) determined the weighted values 14 (exemplar calculations provided in a spreadsheet, Supplemental Digital Content 4, http://links.lww.com/JPHO/A665).Neutropenia was defined as an ANC of ≤ 0.5×10 9 /L.Duration of neutropenic episodes refers to the interval (in weeks) for recovery of ANC from ≤ 0.5×10 9 /L to ≥ 0.75×10 9 /L.The median and interquartile ranges are used to report neutropenia episodes and the weighted mean of drug doses and drug DIs.

Distance and Cost Analysis
The minimum cost incurred by each patient to travel the minimum distance to the hospital and back was computed at a cost of ₹3/km. 15The total minimum cost of MT was estimated based on the patient's total OP visits, and the calculations were extended to include the family (1 child and 2 adults).

Patient Satisfaction Survey
A simplified non-validated telephone survey was used to record the experience of families with the e-clinic MT service.The interview used a structured questionnaire (see survey, Supplemental Digital Content 5, http://links.lww.com/JPHO/A666) to evaluate user satisfaction, service preference (e-clinic alone, in-person alone or the era 4 hybrid model), and costs associated with hospital visits and email consultations.Telephone interviews were performed by members of the clinical team with interviews restricted to parents/caregivers who had accessed the e-clinic on at least 5 occasions, as well as accompanied patients during in-person visits.

Statistical Analysis
Continuous variables are reported as the median interquartile range.Comparisons across groups were performed using the Kruskal-Wallis rank sum test, and pairwise comparisons were performed using the Wilcoxon rank sum test with the Bonferroni correction.Categorical variables are reported as absolute values and proportions (%).All statistical analyses were performed using R statistical software (version 4.1.3)and RStudio (version 2022.02.2).All statistical tests were 2 tailed with P < 0.05 considered significant.

RESULTS
A total of 478 patients were analyzed-163, 116, 68, and 131 in eras 1, 2, 3, and 4, respectively.Era 4 was subdivided into 2: 69 patients who received all 8 MT cycles and 62 patients with a minimum of 4 completed MT cycles.Patient characteristics are described in Table 1.

MT e-clinics Decrease Hospital Visits and OP Workload
In era 1, we did not offer a regular email clinic.Seventeen percent (28/163) of patients who lived further afield did send emails, but for most this was occasional, and the median emails sent per patient was 0 (0 to 0).In era 2, 72% (83/116) requested email clinics and the median was 10 (0 to 17) email consultations per patient.To handle the workload, personnel were designated to handle approximately 12 email consultations per week.In era 3, with the pandemic severely curtailing travel, all patients were managed through the e-clinic.A formal twice-weekly e-clinic was developed with a resident, consultant, and data manager dealing with a median 44 email visits per clinic.Post-pandemic in era 4, we formalized this as a hybrid service, offering patients 4 e-clinics, 1 OP visit, and 1 day-care visit for intrathecal MTX in each 12-week MT cycle.This equates to 32 e-clinics, 8 OP visits, and 8 day-care visits during MT.Patients who live closer to the hospital often choose to come to OP more regularly and some patients, either due to compliance or tolerance issues, require more frequent OP visits bringing the median number of email clinics per patient to 33 (30 to 37) (era 4 patients who completed MT) with 47 (35 to 52) emails per clinic (Table 2).
To better understand the adoption of email clinics by patients, we divided patients in each era into 2 cohorts, a "virtual" cohort, whose number of e-clinic visits was above the median in the respective era, and a "direct" cohort whose number of e-clinic visits were below or equaled the median in that era.As shown in Table 2, of the expected 48 clinic visits in era 4, "virtual" patients recorded a median 56 visits (e-clinics: 38 [35 to 42]; in-person visits: 18 [13 to 22]), with the additional visits: required for weekly supervision during neutropenia episodes."Direct" patients recorded a 4:2 ratio of e-clinic to OP visits (median e-clinics: 30 [28 to  32]; median in-patient visits: 18 [13-24]) highlighting that these patients too were using the e-clinic more often than OP reviews.With the frequency of OP visits halved, the number of patients in each OP clinic reduced considerably.The OP clinic now sees 40 to 50 MT patients each time, down from 70 to 80 patients per clinic in eras 1 to 2.

Improved Dose Optimization With e-clinic and Hybrid Clinics
For each era, the weighted mean DIs of 6MP and MTX and the weighted mean ANC over the entire course of MT for each patient were calculated to determine the median DIs and ANCs in each era cohort.DIs were significantly lower in era 1 compared with eras 2 to 4 (P = 0.0004 and <0.0001 for 6MP and MTX DIs, respectively) (Table 3).Dose titration practice was optimized in era 2, and as a consequence, 6MP and MTX DIs increased significantly (P = 0.0009 and <0.0001, respectively, see table, Supplemental Digital Content 6, http://links.lww.com/JPHO/A667).These improvements in DIs were maintained in era 3 despite the COVID-19 pandemic but with wider DI variations since MT clinics were less frequent due to pandemic restrictions.These variations stabilized in era 4 as clinics were regularized and families traveled or emailed for MT dose consultations.Overall, there were no differences in DIs during eras 2 to 4 (P = 0.43 and 0.34 for 6MP and MTX DIs, respectively).Despite the increase in drug DIs, the median weighted mean ANCs did not differ significantly across the eras (Table 3).Optimization of dose titration in era 2 was accompanied by an increase in the frequency and duration of neutropenia episodes (P = 0.0046 and 0.0286, respectively) compared with era 1 (see table, Supplemental Digital Content 6, http://links.lww.com/JPHO/A667),but declined in eras 3 and 4 (Table 3).There were no differences in DIs and neutropenia episodes between the "virtual" and "direct" groups (see table, Supplemental Digital Content 7, http://links.lww.com/JPHO/A668).
With the hybrid approach in era 4, drug DIs were increased without an increase in hematological toxicities.The wmANC was 1.8 (1.5 to 2.1) while the target ANC was between 1.5 to 0.75, suggesting room for further dose optimization (based on the "treat to tolerance" principle).The increased hematological toxicity in era 2 reflects a learning curve with dose titration practice and structured monitoring.Our experience suggests that parents are quick to inform complications or toxicities by email (see table, Supplemental Digital Content 8, http://links.lww.com/JPHO/A669) and as a result, are seen in the clinic if required.

E-clinics Decrease MT Treatment Costs
We hypothesized that families' preference for e-clinics was related to the distance they had to travel for OP visits.Residence postal codes were available for 332/478 (70%) patients who had completed MT.With the widening of the catchment area, the median distance traveled to the hospital increased from eras 1 to 4 (see table a, Supplemental Digital Content 9, http://links.lww.com/JPHO/A670).In era 4, families who lived within a median of 48 km from the hospital were more likely to opt for in-person appointments.For families living within a 100 km range, the availability of public transport allows clinic attendance as a day visit.Families residing beyond this distance needed to spend at least 1 night in the city when attending MT clinics.From era 2 onward, families living ≥ 100 km were all choosing e-clinics (see table b, Supplemental Digital Content 9, http:// links.lww.com/JPHO/A670).
The current era 4 hybrid approach entails an ~50% decrease in OP costs as the cumulative OP visits over the course of MT decreased to 16 in era 4 compared with the earlier 48 OP visits (Table 4).The decrease in OP visits in era 4 results in aggregate costs savings of ~₹18,000 (~£180), considering that 1 OP visit during MT costs ₹560 (~£5.50, including cost of the CBC test [₹410, ~£4.00] and OP consultation charge [₹150, ~£1.50]) (see tables a-c, Supplemental Digital Content 10, http://links.lww.com/JPHO/A671).

The Hybrid MT Service Elicits Favorable Feedback From Patients and Families
A telephone survey was carried out with a pre-tested questionnaire to assess the acceptance of the hybrid clinic  5).The majority of families (133/156, 85%) were very satisfied with the MT clinic format and felt that the advice received was timely and all additional queries were answered.There were no reported issues in understanding the advice sent by email.Nearly a third of families reported that they would be happier with more (ie, 3) OP visits per cycle, and no family wanted a full e-clinic-based MT service.Nearly all families reported that OP visits were more stressful and disrupted the child's schooling and social life and the majority agreed that the introduction of e-clinics was consequently less demanding on the child and families.

DISCUSSION
The period of MT in childhood ALL lasts for around 2 years during which dosing is personalized for each patient based on serial blood counts.Over the last decade, we have worked to develop a system to create a simplified and standardized approach acceptable to patients and medical staff.In era 1, we introduced patient-held records of doses prescribed and the accompanying blood counts entered by hand at each clinic visit (see spreadsheet, Supplemental Digital Content 2, http://links.lww.com/JPHO/A663).Patients came with this record each time to the clinic.Gradually e-clinics (era 2) were introduced for families residing far away from hospital.During the pandemic (era 3), with patients unable to come to the hospital, we created digitized sheets, which are present on our server and contain past and current records.In era 4, these sheets are filled in the e-clinic and in the OP clinic, with patients receiving a printed-out copy whenever they come to the hospital.The e-clinics progressed from contributing only one-fifth to twothirds of the total MT visits over the years and is the standard norm for ALL patients at TMC.Since January 2021, there are twice weekly e-clinics.With no missing data and careful scrutiny, the weighted mean dose was intensified without an increase in toxicity.The telephonic survey was conducted by the clinical team known to the families, potentially introducing a favorable bias.Nevertheless, results of the telephone survey suggest that families greatly preferred the hybrid approach, though some recommended also having a 24-hour dedicated hotline for MT.
A major concern for patients in low-middle-income countries is the financial toxicity during cancer therapy. 16,17e have previously reported that OP costs constitute 40% to 60% of ALL treatment expense, 18 and with the current hybrid model, the OP visit cost is reduced by ~50%.Subsequently, we believe Out-of-pocket expenses for food and lodging incurred during hospital visits will be reduced.
0][21] Although successful at TMC, the hybrid approach presents certain challenges.E-clinics require additional manpower and time allocation.While no patient refused email clinic, patients from neighboring countries were occasionally irregular with emails.We sent all emails and prescriptions in English.Families replied in English or in vernacular.Many families used internet-cafés to email the blood reports.To date, we have not encountered any misinterpretation of instructions, and families report the assistance of other family members or internet-café employees when required.If there is a doubt, clinical unit members have called parents to clarify.Medication adherence is a critical factor during MT. 22,23Though this does not appear to be a major problem at our center, this is difficult to diagnose without a clinic visit.It is possible that some toxicities such as hypoglycemia may be under-reported by parents. 19We  educate parents on the expected toxicities at the start of MT.These are reported by parents over email and addressed immediately or in the next OP clinic.One challenge we did face was the reliability of blood counts done elsewhere due to varying lab standards.This affected a few patients whose counts done locally differed widely from those done at TMC.In all cases, we were able to resolve this quickly by working with the families to identify alternative laboratories.Chemotherapy drugs are not widely available in small rural areas and patients are prescribed sufficient medication to take home with them when they visit OP.During the pandemic, this did pose a problem for some patients who were unable to procure supplies locally, though distributors did try their best to directly deliver to their homes.The digitized longitudinal data eases optimal dose titration practice.In the future, additional information can be included to assist in treating inherent drug-sensitive patients with protocol modulation 3 or adjuvant therapy. 246][27] This will facilitate the standardization of MT therapy across centers in countries the size of India.Patients can upload the CBC report and receive back drug prescriptions with a possible visual aid of daily dose distribution and a reminder message on their mobile phones to take the drugs.With these data stored on a database, machine learning/artificial intelligence tools can further optimize dosing 28 and aid clinicians.
children receiving MT attended OP at least once in 2 weeks.In era 2 (January 26, 2018, to March 25, 2020) email consultations (e-clinic) were introduced, initially selectively for patients living far away from TMC and later for all patients.Era 3 (March 26, 2020, to May 14, 2021) marked the SARS-CoV-2 pandemic, requiring e-clinic MT management of all patients in this era as most patients were unable to travel to the hospital.The adaptations in era 3 led to the introduction of a hybrid MT service model in era 4 (May 15, 2021, to February 28, 2023) for all patients, consisting of a minimum of e-clinic and 2 inperson visits in each 12-week MT cycle.Era 4 comprises of 2 subgroups: patients who have completed MT and those who have completed a minimum 4 cycles of MT.Patients who were treated across eras were assigned to the era where > 50% of MT was administered.

TABLE 1 .
Patient Characteristics

TABLE 3 .
Treatment and Treatment Response in Different MT Eras